{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Analyzing UVES Spectroscopy with Astropy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Authors\n",
    "Moritz Guenther, Miguel de Val-Borro, Emily Rice (Learning Goals & formatting)\n",
    "\n",
    "## Learning Goals\n",
    "* *Input/Output*: Read in spectral data from a list of FITS files using [astropy.io](http://docs.astropy.org/en/stable/io/fits/index.html) and [astropy.wcs](http://docs.astropy.org/en/stable/wcs/index.html)\n",
    "* *Coding*: Make code reusable as a function\n",
    "* *Analysis*: Analyze spectral data using [astropy.units](http://docs.astropy.org/en/stable/units/index.html),  [astropy.constants](http://docs.astropy.org/en/stable/constants/index.html), and [astropy.time](http://docs.astropy.org/en/stable/time/index.html)\n",
    "* *Input/Output*: Generate a LaTeX table using [astropy.table](http://docs.astropy.org/en/stable/table/index.html)\n",
    "* *Visualization*: Plot spectral data using [matplotlib](https://matplotlib.org/)\n",
    "\n",
    "## Keywords\n",
    "astropy.io, astropy.wcs, astropy.units, astropy.constants, astropy.time, astropy.table, matplotlib, spectra, UVES\n",
    "\n",
    "## Summary\n",
    "\n",
    "This tutorial follows our real-life data analysis of MN Lup and the code developed\n",
    "below is taken (with only minor modifications) from the code that we used to\n",
    "prepare the publication. The plots that we will develop below\n",
    "appear in very similar form in the article published in\n",
    "[ApJ, 771, 1, 70](http://adsabs.harvard.edu/abs/2013ApJ...771...70G).\n",
    "\n",
    "The examples below depend on each other and the plots in the last section make\n",
    "use of things calculated in the earlier sections. Thus, if you need to restart\n",
    "your python session in the course of this tutorial, please execute all the code\n",
    "again."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Before you proceed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set up matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Download the tar file and extract the files by executing the python code below. Executing this block will download and extract a tar file with data necessary for this tutorial. By default it will run in whatever directory you have placed this notebook. If that's some place you don't want to fill with data files, change the `working_dir_path` variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tarfile\n",
    "from astropy.utils.data import download_file\n",
    "url = 'http://data.astropy.org/tutorials/UVES/data_UVES.tar.gz'\n",
    "f = tarfile.open(download_file(url, cache=True), mode='r|*')\n",
    "working_dir_path = '.'  # CHANGE TO WHEREVER YOU WANT THE DATA TO BE EXTRACTED\n",
    "f.extractall(path=working_dir_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scientific background"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this tutorial we analyze data from MN Lup, a T Tauri star in the\n",
    "Taurus-Auriga star forming region located at a distance of about 140 pc. MN Lup\n",
    "has been observed simultaneously with XMM-Newton and the UVES spectrograph on the\n",
    "VLT. MN Lup is suspected to be a classical T Tauri star, that is accreting mass\n",
    "from a circumstellar disk. MN Lup has been Doppler imaged by\n",
    "[Strassmeier et al. 2005](http://adsabs.harvard.edu/abs/2005A%26A...440.1105S)\n",
    "with a very similar UVES setup and those authors claim an rotationally modulated\n",
    "accretion spot.\n",
    "\n",
    "In the X-ray data we find moderate indications for accretion. In this\n",
    "tutorial we analyze (some of) the UVES data to search for rotationally modulated\n",
    "features in the emission line profiles, which could be due to an accretion spot\n",
    "on the stellar surface."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reading the data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A previous Astropy tutorial already covered\n",
    "[handling FITS files](../FITS-header.html) and WCS transformations, so the explanation here\n",
    "is only very brief. Check the [astropy documentation](http://docs.astropy.org)\n",
    "or the other two tutorials for more details:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from glob import glob\n",
    "import os\n",
    "\n",
    "import numpy as np\n",
    "\n",
    "from astropy.wcs import WCS\n",
    "from astropy.io import fits\n",
    "\n",
    "# os.path.join is a platform-independent way to join two directories\n",
    "globpath = os.path.join(working_dir_path, 'UVES/*.fits')\n",
    "\n",
    "print(globpath)\n",
    "# glob searches through directories similar to the Unix shell\n",
    "filelist = glob(globpath)\n",
    "\n",
    "# sort alphabetically - given the way the filenames are\n",
    "# this also sorts in time\n",
    "filelist.sort()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Read the first FITS file in the list and check what is in there:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sp = fits.open(filelist[0])\n",
    "sp.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We see that the data is given as the primary image and all other info is\n",
    "part of the primary header. So, we can extract the WCS from that header\n",
    "to get the wavelength coordinate.\n",
    "If you see warnings about a non-standard RADECSYS, don't worry\n",
    "about this -- the WCS will still work, it just doesn't\n",
    "fully conform to the WCS standard."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "header = sp[0].header\n",
    "\n",
    "wcs = WCS(header)\n",
    "#make index array\n",
    "index = np.arange(header['NAXIS1'])\n",
    "\n",
    "wavelength = wcs.wcs_pix2world(index[:,np.newaxis], 0)\n",
    "wavelength.shape\n",
    "#Ahh, this has the wrong dimension. So we flatten it.\n",
    "wavelength = wavelength.flatten()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The flux is contained in the primary image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flux = sp[0].data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Making code reusable as a function"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we don't want to repeat this process for every single file by hand,\n",
    "so let's define a function that takes the filename as input and returns\n",
    "the wavelength and flux arrays and the time of the observation.\n",
    "In python, functions are created with the ``def`` statements.\n",
    "All lines that have an indentation level below the `def` statement are part\n",
    "of the function. Functions can (but do not have to) return values using\n",
    "the ``return`` statement.\n",
    "\n",
    "If a function ``func`` is contained in a file called ``spectra_utils.py`` in\n",
    "the current directory, then this file can be imported into a python session in\n",
    "order to use the function `func` with the following command:\n",
    "\n",
    "```\n",
    "import spectra_utils\n",
    "a = spectra_utils.func(param1, param2, ...)\n",
    "```\n",
    "\n",
    "Alternatively, you can import just one (or a few) of many different functions\n",
    "that are defined in your file ``spectra_utils.py``:\n",
    "\n",
    "```\n",
    "from spectra_utils import func\n",
    "a = func(param1, param2, ...)\n",
    "```\n",
    "\n",
    "You'll recognize that python does not make a difference between modules that come\n",
    "with python (e.g. `glob`), external modules (e.g. `numpy` or `astropy`) and modules\n",
    "that you write yourself. The syntax to import those modules or functions\n",
    "is the same in all cases, provided that the directory where your module is\n",
    "defined is in the search path [more about python modules and the search path](http://docs.python.org/2/tutorial/modules.html)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once you used ``import spectra_utils`` python will not monitor the source file.\n",
    "If you change the source code of ``func`` in the file, you'll need to\n",
    "``reload(spectra_utils)`` to load the new version of ``func``.\n",
    "\n",
    "So, after all this discussion, we can now define a function that automates the\n",
    "loading of a single spectrum using the commands we developed above. Even if\n",
    "this function is fairly short, we still add some documentation to the header,\n",
    "so that we can look up what parameters it needs when we come back to this\n",
    "project a while later. We like to comment every function that is longer\n",
    "than two lines."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_spec(filename):\n",
    "    '''Read a UVES spectrum from the ESO pipeline\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    filename : string\n",
    "    name of the fits file with the data\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    wavelength : np.ndarray\n",
    "    wavelength (in Ang)\n",
    "    flux : np.ndarray\n",
    "    flux (in erg/s/cm**2)\n",
    "    date_obs : string\n",
    "    time of observation\n",
    "    '''\n",
    "    sp = fits.open(filename)\n",
    "    header = sp[0].header\n",
    "\n",
    "    wcs = WCS(header)\n",
    "    #make index array\n",
    "    index = np.arange(header['NAXIS1'])\n",
    "\n",
    "    wavelength = wcs.wcs_pix2world(index[:,np.newaxis], 0)\n",
    "    wavelength = wavelength.flatten()\n",
    "    flux = sp[0].data\n",
    "\n",
    "    date_obs = header['Date-OBS']\n",
    "    return wavelength, flux, date_obs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try to find out how you can read the help for this function from the\n",
    "command line."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "help(read_spec)\n",
    "# or\n",
    "read_spec?\n",
    "# In the IPython notebook, the easiest way to see the help for a function is to type read_spec<TAB>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset of UVES spectra should have been taken using all the same setup.\n",
    "Write a function that returns the exposure time (``EXPTIME``),\n",
    "the wavelength zero point\n",
    "(``CRVAL1``), and the arm used (UVES has a red and a blue arm - see keyword\n",
    "``HIERARCH ESO INS PATH``). Then check that all exposures have the same\n",
    "setup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def read_setup(filename):\n",
    "    '''Get setup for UVES spectrum from the ESO pipeline\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    filename : string\n",
    "    name of the fits file with the data\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    exposure_time : float\n",
    "    wavelength_zero_point : float\n",
    "    optical_arm : string\n",
    "    '''\n",
    "    sp = fits.open(filelist[0])\n",
    "    header = sp[0].header\n",
    "\n",
    "    return header['EXPTIME'], header['CRVAL1'], header['HIERARCH ESO INS PATH']\n",
    "\n",
    "# Let's just print the setup on the screen\n",
    "# We'll see if it's all the same.\n",
    "for f in filelist:\n",
    "    print(read_setup(f))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The UVES pipeline that was used to reduce the data that we use in the this example\n",
    "employs a fixed wavelength grid (see exercise above),\n",
    "thus the ``wavelength`` is the same for all spectra.\n",
    "This makes it easy to define an array that can hold the fluxes of all\n",
    "observations. Then, we loop over the list of all filenames and fill this array\n",
    "with data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "flux = np.zeros((len(filelist), len(wavelength)))\n",
    "# date comes as string with 23 characters (dtype = 'S23')\n",
    "date = np.zeros((len(filelist)), dtype = 'U23')\n",
    "\n",
    "for i, fname in enumerate(filelist):\n",
    "    w, f, date_obs = read_spec(fname)\n",
    "    flux[i,:] = f\n",
    "    date[i] = date_obs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Units and constants in astropy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Often, one has to keep track of the units for certain values. Was the wavelength\n",
    "given in Angstrom or in nm? In X-ray observations, a common unit of wavelength is\n",
    "keV. How many nm is 0.65 keV?\n",
    "[`astropy.units`](http://docs.astropy.org/en/stable/units/index.html)\n",
    "offers a framework that can take\n",
    "care of this book-keeping and propagate the units through many (but not all)\n",
    "mathematical operations (e.g. addition, division, multiplication).\n",
    "Furthermore,\n",
    "[`astropy.constants`](http://docs.astropy.org/en/stable/constants/index.html)  supplies the values of\n",
    "many physical and astronomical constants.\n",
    "The easiest way to attach a unit to a number is by multiplication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import astropy.units as u\n",
    "from astropy.constants.si import c, G, M_sun, R_sun\n",
    "\n",
    "wavelength = wavelength * u.AA\n",
    "\n",
    "# Let's define some constants we need for the exercises further down\n",
    "# Again, we multiply the value with a unit here\n",
    "heliocentric = -23. * u.km/u.s\n",
    "v_rad = -4.77 * u.km / u.s  # Strassmeier et al. (2005)\n",
    "R_MN_Lup = 0.9 * R_sun      # Strassmeier et al. (2005)\n",
    "M_MN_Lup = 0.6 * M_sun      # Strassmeier et al. (2005)\n",
    "vsini = 74.6 * u.km / u.s   # Strassmeier et al. (2005)\n",
    "period = 0.439 * u.day      # Strassmeier et al. (2005)\n",
    "\n",
    "inclination = 45. * u.degree # Strassmeier et al. (2005)\n",
    "# All numpy trigonometric functions expect the input in radian.\n",
    "# So far, astropy does not know this, so we need to convert the\n",
    "# angle manually\n",
    "incl = inclination.to(u.radian)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can use those variables in our calculations. MN Lup is a T Tauri\n",
    "star (TTS), which is possibly surrounded by an accretion disk. In the spectra\n",
    "we'll be looking for signatures of accretion. We expect those accretion\n",
    "signatures to appear close to the free-fall velocity v that a mass m reaches, when\n",
    "it hits the stellar surface. We can calculate the infall speed using simple\n",
    "energy conservation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$$E_{kin}  =  E_{grav}$$\n",
    "$$\\frac{1}{2} m v^2  =  G \\frac{m M_*}{R_*}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, let's calculate the free-fall velocity for MN Lup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "v_accr = (2.* G * M_MN_Lup/R_MN_Lup)**0.5 \n",
    "print(v_accr)\n",
    "# Maybe astronomers prefer it in the traditional cgs system?\n",
    "print(v_accr.cgs)\n",
    "# Or in some really obscure unit?\n",
    "from astropy.units import imperial\n",
    "print(v_accr.to(imperial.yd / u.hour))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "How does the accretion velocity relate to the rotational velocity?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "v_rot = vsini / np.sin(incl)\n",
    "v_accr / v_rot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Oh, what is that? The seconds are gone, but ``astropy.quantity`` objects keep\n",
    "their different length units unless told otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(v_accr / v_rot).decompose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The reason for this is that it's not uncommon to use different length units in\n",
    "a single constant, e.g. the Hubble constant is commonly given in \"km/ (s Mpc)\".\n",
    "\"km\" and \"Mpc\" are both units of length, but generally you do *not* want to\n",
    "shorten this to \"1/s\".\n",
    "\n",
    "We can now use the ``astropy.units`` mechanism to correct the wavelength\n",
    "scale to the heliocentric velocity scale."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "$\\lambda_{heliocentric} = \\lambda_{bariocentric} * (1 + \\frac{v_{helio}}{c})$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We could try:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wavelength = wavelength * (1. + heliocentric/c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "However, this fails, because ``heliocentric/c`` is in units of \"km/m\" and ``1.``\n",
    "is just a number. From the notation above, it is not clear what we actually want.\n",
    "Do we ask for the value of ``heliocentric/c + 1.`` or do we want to simplify the\n",
    "units of  ``heliocentric/c`` and after that add ``1.``?\n",
    "There are several ways to make the instruction precise,\n",
    "but one is to explicitly add ``u.dimensionless_unscaled`` to ``1.``\n",
    "to tell astropy that this number is dimensionless and does not carry any scaling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wavelength = wavelength * (1. * u.dimensionless_unscaled+ heliocentric/c)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We want to mention one more feature here (check out\n",
    "[`astropy.units`](http://docs.astropy.org/en/stable/units/index.html) for\n",
    "more): The ability to convert the spectral axis to frequencies or energies.\n",
    "Normally, a unit of length is not equivalent to a unit of energy or to a\n",
    "frequency, but this conversion makes sense for the wavelength of a spectrum.\n",
    "This is how it can be done:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wavelength.to(u.keV, equivalencies=u.spectral())\n",
    "wavelength.to(u.Hz, equivalencies=u.spectral())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Spectroscopically, MN Lup is classified as spectral type M0 V, thus\n",
    "the gravitational acceleration on the surface $\\log(g)$\n",
    "should be comparable to the sun.\n",
    "(For non-stellar astronomers: Conventionally, all values are given\n",
    "in the cgs system. The value for the sun is $\\log(g) = 4.4$.)\n",
    "\n",
    "Calculate $\\log(g)$ for MN Lup with the values for the mass\n",
    "and radius given above. Those values were determined from\n",
    "evolutionary tracks. Check if the $\\log(g)$ is consistent\n",
    "with the value expected from spectroscopy."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The values from evolutionary tracks are indeed consistent with the\n",
    "spectroscopically estimated surface gravity."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(np.log10((G*M_MN_Lup/R_MN_Lup**2)/u.cm*u.second**2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write a function that turns a wavelength scale into a velocity scale.\n",
    "We want to input a wavelengths array and the rest wavelength of a spectral\n",
    "line. We need this function later to show the red- and blueshift of the\n",
    "spectrum relative to the the Ca II H line. Use the following definition\n",
    "to make sure that the code below can use it later.\n",
    "You can test if your function works by calculating the Doppler shift\n",
    "of the following wavelengths relative to $H_\\alpha$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "waveclosetoHa = np.array([6562.,6563,6565.]) * u.AA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We get -132, -86 and +5 km/s."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This function uses the Doppler equivalency between wavelength and velocity\n",
    "import astropy.units as u\n",
    "def wave2doppler(w, w0):\n",
    "    w0_equiv = u.doppler_optical(w0)\n",
    "    w_equiv = w.to(u.km/u.s, equivalencies=w0_equiv)\n",
    "    return w_equiv\n",
    "\n",
    "print(wave2doppler(waveclosetoHa, 656.489 * u.nm).to(u.km/u.s))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Write a function that takes a wavelength array and the rest wavelength of\n",
    "a spectral line as input, turns it into a Doppler shift (you can use\n",
    "the function from the last exercise),\n",
    "subtracts the radial velocity of MN Lup (4.77 km/s) and expresses\n",
    "the resulting velocity in units of vsini.\n",
    "We need this function later to show the red- and blueshift of the\n",
    "spectrum relative to the Ca II H line. Use the following definition\n",
    "to make sure the that code below can use it later."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def w2vsini(wavelength_array, wavelength_line):\n",
    "    # .. replace this with your implementation ..\n",
    "    return array_of_shifts_in_vsini"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def w2vsini(w, w0):\n",
    "    v = wave2doppler(w, w0) - 4.77 * u.km/u.s\n",
    "    return v / vsini"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Converting times"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[`astropy.time`](http://docs.astropy.org/en/stable/time/index.html)\n",
    "provides methods to convert times and dates between different\n",
    "systems and formats. Since the ESO FITS headers already contain the time of the\n",
    "observation in different systems, we could just read the keyword in the time\n",
    "system we like, but we will use ``astropy.time`` to make this conversion here.\n",
    "``astropy.time.Time`` will parse many common input formats (strings, floats), but\n",
    "unless the format is unambiguous the format needs to be specified (e.g. a number\n",
    "could mean JD or MJD or year). Also, the time system needs to be given (e.g. UTC).\n",
    "Below are several examples, initialized from different header keywords."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from astropy.time import Time\n",
    "t1 = Time(header['MJD-Obs'], format = 'mjd', scale = 'utc')\n",
    "t2 = Time(header['Date-Obs'], scale = 'utc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Times can be expressed in different formats:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t1\n",
    "t1.isot\n",
    "t2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or be converted to a different time system."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "t1.tt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Times can also be initialized from arrays and we can calculate time differences."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "obs_times = Time(date, scale = 'utc')\n",
    "delta_t = obs_times - Time(date[0], scale = 'utc')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we want to express the time difference between the individual spectra of\n",
    "MN Lup in rotational periods. While the unit of ``delta_t`` is days, unfortunately\n",
    "``astropy.time.Time`` and ``astropy.units.Quantity`` objects don't work together\n",
    "yet, so we'll have to convert from one to the other explicitly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "delta_p = delta_t.value * u.day / period"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Normalize the flux to the local continuum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example we want to look at the time evolution of a single specific\n",
    "emission line in the spectrum. In order to estimate the equivalent width\n",
    "or make reasonable plots we need to normalize the flux to the local continuum.\n",
    "In this specific case the emission line is bright and the continuum can be\n",
    "described reasonably by a second-order polynomial."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So, we define two regions left and right of the emission line, where we fit the\n",
    "polynomial. Looking at the figure, ``[3925*u.AA, 3930*u.AA]`` and\n",
    "``[3938*u.AA, 3945*u.AA]`` seem right for that. Then, we normalize the flux by\n",
    "this polynomial.\n",
    "\n",
    "The following function will do that:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def region_around_line(w, flux, cont):\n",
    "    '''cut out and normalize flux around a line\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    w : 1 dim np.ndarray\n",
    "    array of wavelengths\n",
    "    flux : np.ndarray of shape (N, len(w))\n",
    "    array of flux values for different spectra in the series\n",
    "    cont : list of lists\n",
    "    wavelengths for continuum normalization [[low1,up1],[low2, up2]]\n",
    "    that described two areas on both sides of the line\n",
    "    '''\n",
    "    #index is true in the region where we fit the polynomial\n",
    "    indcont = ((w > cont[0][0]) & (w < cont[0][1])) |((w > cont[1][0]) & (w < cont[1][1]))\n",
    "    #index of the region we want to return\n",
    "    indrange = (w > cont[0][0]) & (w < cont[1][1])\n",
    "    # make a flux array of shape\n",
    "    # (number of spectra, number of points in indrange)\n",
    "    f = np.zeros((flux.shape[0], indrange.sum()))\n",
    "    for i in range(flux.shape[0]):\n",
    "        # fit polynomial of second order to the continuum region\n",
    "        linecoeff = np.polyfit(w[indcont], flux[i, indcont], 2)\n",
    "        # divide the flux by the polynomial and put the result in our\n",
    "        # new flux array\n",
    "        f[i,:] = flux[i,indrange] / np.polyval(linecoeff, w[indrange].value)\n",
    "    return w[indrange], f\n",
    "\n",
    "wcaII, fcaII = region_around_line(wavelength, flux,\n",
    "    [[3925*u.AA, 3930*u.AA],[3938*u.AA, 3945*u.AA]])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Publication ready output"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tables"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll calculate the equivalent width in Angstroms of the emission line\n",
    "for the first spectrum."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ew = fcaII[0,:] - 1.\n",
    "ew = ew[:-1] * np.diff(wcaII.to(u.AA).value)\n",
    "print(ew.sum())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using ``numpy`` array notation we can actually process all spectra at once."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "delta_lam = np.diff(wcaII.to(u.AA).value)\n",
    "ew = np.sum((fcaII - 1.)[:,:-1] * delta_lam[np.newaxis, :], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we want to generate a LaTeX table of the observation times, period\n",
    "and equivalent width that we can directly paste into our manuscript. To do so,\n",
    "we first collect all the columns and make an ``astropy.table.Table`` object. (Please\n",
    "check [`astropy.table`](http://docs.astropy.org/en/stable/table/index.html)\n",
    "or `tabular-data` for more\n",
    "details on ``Table``). So, here is the code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from astropy.table import Column, Table\n",
    "from astropy.io import ascii\n",
    "\n",
    "datecol = Column(name = 'Obs Date', data = date)\n",
    "pcol = Column(name = 'phase', data = delta_p, format = '{:.1f}')\n",
    "ewcol = Column(name = 'EW', data = ew, format = '{:.1f}', unit = '\\\\AA')\n",
    "tab = Table((datecol, pcol, ewcol))\n",
    "# latexdicts['AA'] contains the style specifics for A&A (\\hline etc.)\n",
    "tab.write(os.path.join(working_dir_path, 'EWtab.tex'), latexdict = ascii.latexdicts['AA'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plots"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll make two plots. The plotting is done with\n",
    "[`matplotlib`](http://matplotlib.org), and does not involve Astropy itself.\n",
    "Plotting is introduced in `plotting-and-images` and more details on\n",
    "plotting can be found there. When in doubt, use the search engine of your choice\n",
    "and ask the internet. Here, we mainly want to illustrate that Astropy can be\n",
    "used in real-live data analysis.\n",
    "Thus we don't explain every step in the plotting in detail.\n",
    "The plots we produce below appear in very\n",
    "similar form in Guenther et al. 2013 (ApJ, 771, 70)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In both cases we want the x-axis to show the Doppler shift expressed in units\n",
    "of the rotational velocity. In this way, features that are rotationally\n",
    "modulated will stick out between -1 and +1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = w2vsini(wcaII, 393.366 * u.nm).decompose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we'll show the line profile."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set reasonable figsize for 1-column figures\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "ax.plot(x, fcaII[0,:], marker='', drawstyle='steps-mid')\n",
    "ax.set_xlim([-3,+3])\n",
    "ax.set_xlabel('line shift [v sin(i)]')\n",
    "ax.set_ylabel('flux')\n",
    "ax.set_title('Ca II H line in MN Lup')\n",
    "# when using this interface, we need to explicitly call the draw routine\n",
    "plt.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The plot above shows only a single spectrum. Plot all spectra into a single\n",
    "plot and introduce a sensible offset between them, so that we can follow\n",
    "the time evolution of the line."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are clearly several ways to produce a well-looking plot. Here is one\n",
    "way:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "yshift = np.arange((fcaII.shape[0])) * 0.5\n",
    "#shift the second night up by a little more\n",
    "yshift[:] += 1.5\n",
    "yshift[13:] += 1\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "\n",
    "for i in range(25):\n",
    "    ax.plot(x, fcaII[i,:]+yshift[i], 'k')\n",
    "\n",
    "#separately show the mean line profile in a different color\n",
    "ax.plot(x, np.mean(fcaII, axis =0))\n",
    "ax.set_xlim([-2.5,+2.5])\n",
    "ax.set_xlabel('line shift [$v \\\\sin i$]')\n",
    "ax.set_ylabel('flux')\n",
    "ax.set_title('Ca II H line in MN Lup')\n",
    "fig.subplots_adjust(bottom = 0.15)\n",
    "plt.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we'll make a more advanced plot. For each spectrum we calculate\n",
    "the difference to the mean flux."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fmean = np.mean(fcaII, axis=0)\n",
    "fdiff = fcaII - fmean[np.newaxis,:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following simple plot, we can already see features moving through the line.\n",
    "However, the axis scales are not right, the gap between both nights is not visible\n",
    "and there is no proper labeling."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "im = ax.imshow(fdiff, aspect = \"auto\", origin = 'lower')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the following, we'll plot the spectra from both nights separately.\n",
    "Also, we'll pass the ``extent`` keyword to ``ax.imshow`` which takes care\n",
    "of the axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ind1 = delta_p < 1 * u.dimensionless_unscaled\n",
    "ind2 = delta_p > 1 * u.dimensionless_unscaled\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "\n",
    "for ind in [ind1, ind2]:\n",
    "    im = ax.imshow(fdiff[ind,:], extent = (np.min(x), np.max(x), np.min(delta_p[ind]), np.max(delta_p[ind])), aspect = \"auto\", origin = 'lower')\n",
    "\n",
    "ax.set_ylim([np.min(delta_p), np.max(delta_p)])\n",
    "ax.set_xlim([-1.9,1.9])\n",
    "plt.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, this plot is already much better, but there are still some things that can be\n",
    "improved:\n",
    "\n",
    "* Introduce an offset on the y-axis to reduce the amount of white space.\n",
    "* Strictly speaking, the image shown is not quite the right scale because the\n",
    "``extent`` keyword gives the edges of the image shown, while ``x`` and\n",
    "``delta_p`` contain the bin mid-points.\n",
    "* Use a gray scale instead of color to save publication charges.\n",
    "* Add labels to the axis.\n",
    "\n",
    "The following code addresses these points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# shift a little for plotting purposes\n",
    "pplot = delta_p.copy().value\n",
    "pplot[ind2] -= 1.5\n",
    "# image goes from x1 to x2, but really x1 should be middle of first pixel\n",
    "delta_t = np.median(np.diff(delta_p))/2.\n",
    "delta_x = np.median(np.diff(x))/2.\n",
    "# imshow does the normalization for plotting really well, but here I do it\n",
    "# by hand to ensure it goes -1,+1 (that makes color bar look good)\n",
    "fdiff = fdiff / np.max(np.abs(fdiff))\n",
    "\n",
    "fig = plt.figure()\n",
    "ax = fig.add_subplot(1,1,1)\n",
    "\n",
    "for ind in [ind1, ind2]:\n",
    "    im = ax.imshow(fdiff[ind,:],\n",
    "    extent = (np.min(x)-delta_x, np.max(x)+delta_x,\n",
    "    np.min(pplot[ind])-delta_t, np.max(pplot[ind])+delta_t),\n",
    "    aspect = \"auto\", origin = 'lower', cmap = plt.cm.Greys_r)\n",
    "\n",
    "ax.set_ylim([np.min(pplot)-delta_t, np.max(pplot)+delta_t])\n",
    "ax.set_xlim([-1.9,1.9])\n",
    "ax.set_xlabel('vel in $v\\\\sin i$')\n",
    "ax.xaxis.set_major_locator(plt.MaxNLocator(4))\n",
    "\n",
    "def pplot(y, pos):\n",
    "    'The two args are the value and tick position'\n",
    "    'Function to make tick labels look good.'\n",
    "    if y < 0.5:\n",
    "        yreal = y\n",
    "    else:\n",
    "        yreal = y + 1.5\n",
    "    return yreal\n",
    "\n",
    "formatter = plt.FuncFormatter(pplot)\n",
    "ax.yaxis.set_major_formatter(formatter)\n",
    "ax.set_ylabel('period')\n",
    "fig.subplots_adjust(left = 0.15, bottom = 0.15, right = 0.99, top = 0.99)\n",
    "plt.draw()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Understand the code for the last plot. Some of the commands used are\n",
    "already pretty advanced stuff. Remember, any internet search engine can be\n",
    "your friend."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Clearly, we didn't develop this code for scratch.\n",
    "The [matplotlib gallery](http://matplotlib.org/gallery.html) is our\n",
    "preferred place to look for plotting solutions."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Contributing to Astropy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Astropy](http://astropy.org) is an open-source and community-developed\n",
    "Python package, which means that is only as good as the contribution of the\n",
    "astronomical community. There will always people who have more fun writing\n",
    "code and others who have more fun using it. However, if you find a bug and do not\n",
    "report it, then it is unlikely to be fixed. If you wish for a specific feature,\n",
    "then you can either implement it and contribute it or at least fill in a feature\n",
    "request.\n",
    "\n",
    "If you want to get help or discuss issues with other Astropy users, you can\n",
    "sign up for the [astropy mailing list](http://mail.scipy.org/mailman/listinfo/astropy).\n",
    "Alternatively, the [astropy-dev](http://groups.google.com/group/astropy-dev) list is where you should go to\n",
    "discuss more technical aspects of Astropy with the developers.\n",
    "\n",
    "If you have come across something that you believe is a bug, please open a\n",
    "ticket in the Astropy [issue tracker](http://github.com/astropy/astropy/issues), and we will look into it\n",
    "promptly.\n",
    "\n",
    "Please try to include an example that demonstrates the issue and will allow the\n",
    "developers to reproduce and fix the problem.  If you are seeing a crash\n",
    "then frequently it will help to include the full Python stack trace as well as\n",
    "information about your operating system (e.g. MacOSX version or Linux version)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "astropy-tutorials": {
   "author": "Hans Moritz Günther <hguenther@cfa.harvard.edu>, Miguel de Val-Borro <valborro@princeton.edu>",
   "date": "August 2014",
   "description": "This tutorial demonstrates the data analysis of MN Lup obtained with the UVES spectrograph on the VLT.",
   "link_name": "Analyzing UVES Spectroscopy with Astropy",
   "name": "",
   "published": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
